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ABSTRACT 

A study was done examining the experiences of four 
California community colleges that received pilot program grants to 
participate in a legislatively mandated, statewide accounta):9ility 
program. The participating colleges were Hount San Jtotonio College 
(Los Angeles) f San Joaquin Delta College (Stockton), San 
Jose-Evergreen, and Santa Barl>ara City College. The data collection 
included centralized collection and reporting of information on 
student access, student success, student satisfaction, staff 
composition, and fiscal condition. Also as part of the study, the 
colleges and the state Chancellor's office developed a guide for 
local use and a systemwide accountability progr€un. The study found 
that accountability had two meanings: at the state level it was tied 
to educational finance and the desire to make policy based on 
objective, defensible information- At the local, college level, it 
was linked to institutional effectiveness. The findings also seemed 
to suggest that, though features of the state accountability program 
are similar to recommendations made in Total Quality Management r 
colleges did not embrace that approach. Conclusions emphasiz«a the 
importance of evaluation and assessment comprising multiple sources 
and types of information for a composite picture. Included are 23 
references. (JB) 
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Abstract 



Recent legislation mandated a statewide accountability program for California's 
Community Colleges. This study describes the experiences of four colleges that received 
pilot program grants. The main finding was that accountability had two meanings. At the 
state level it ties to educational finance, and the desire to make policy based on objeaive. 
defensible information. At the local, college level it linb to institutional effectiveness. 
Many of California's colleges currently lack the technical resources for an accountability or 
institutional effectiveness program. A plan to improve readiness for accountability 
included basic data collection and reponing, in-depth accountability studies, surveys of 
student satisfaction and long range program effeaiveness, improved data collection, and a 
dearirighouse. 
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Tough Love or Cooperative Learning? 
Accountability for Higher Education 



Introduction 

"Semantic aphasia" is a state of mind that results from the mantra-like recitation of a word 
or phrase — it loses meaning. This sometimes appears true tor "accountability." This 
study describes the stru^le to define "accountability" by a group of community coU^e 
educators, including students, faculty, administrators, and trustees. A central dilemma in 
creating this definition related to the role of the statewide Chancellor s OfHcc as either a 
tough (love) overseer, or as a provider of tedinical assistance and leadership (cooperative 
learning) for local institutional effeaiveness programs. According to one view the 
Chancellor's Office would adopt the role of a coach, trying to improve the teiim's 
performance. 

"By design and by talent," wrote basketball player Bill Russell of his team, the 
Boston Celtics, "we were a team of specialists, and like a team of specialists in any 
field, our performance depended both on individual excellence and how well we 
worked together. None of us had to strain to undented that we had to 
complement each othere' specialties; it was simply a faa, and we ail tried to figure 
out ways to make our combination more effeaivc." (Peter M. Senge, p. 233) 

A very different view of accountability may stem from public concerns about the quality 
of higher education ~ :itudent loan defaults, the difficulty of obtaining required courses, or 
the length of time needed for completion. According to this other view, the Chancellor's 
OfBce would use information to satisfy the public's right to know, centralize control, and 
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require improvements. 

"The environments of colleges and universities are increasingly characterized by 
turbulence, competitiveness, lean resources, unpredictability, and periodic decline . 
. ... when organizations face conditions they define as threatening, the tendenr;- is to 
become rigid - for example, centralized, conservative, protective, inflexible, and 
non adaptable. ..." (Cameron & Tschirart, p. 88) 

M ilieu 

A sweeping legislative reform of the California Community Colleges in 1988 provided 
the system with $140 million in new funds and requited the state Board to develop and 
implement a comprehensive educational and fiscal accountability s)^tcm. (Board of 
Governors, 1989; MacDougall and Friedlander, 1990) The mandate was "to produce a 
published repon of California Community Coll^ accountability," that should cover such 
topics as student access, transfor to senior institutions, achievement, saiisfaaion, program 
completion, occupational preparation, basic skills, English as a second language, diversity, 
and fiscal conditions. 

Policymakers debated the issues of funding for education and accountability at this time. 
By a narrow margin, voters amended the sute's constitution in 1988 to establish a 
minimum level of state funding for schools and woninunity colleges. This school funding 
initiative, known as Proposition 98, also required die prqjaration of "School 
Accountability Report Cards" for K- 12 schools, but not for community colle^. 
(California Secretary of State, 1988). Each public school district board had to issue an 
annual report for every school covering student achievement, dropouts, expenditures, class 
size, teacher assignment, textbooks, counseling, substitute teachers, safety, teacher 
evaluation, classroom climate, staff development, and leadership. 
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The governance of California's Community College System is certainly relevant to the 
task of defining accountability. (Commission for the Review of the Master Plan for 
Higher Education, 1987; Board of Governors, 1989). The decentralized nature of 
community a>Ilege governance may limit the geneializability of this study to other states. 
There are 71 districts, including 107 colleges, which serve about 1.3 million students. At 
the st^e level the Board of Governors provide leadership and direction in the 
development of the system while maintaining as much local authority and control as 
possible. Local governing boards establish, maintain and operate locd colleges aarording 
to law. There is no formal linkage between the Board of Governors and local governing 
boards or between local chief administrative oflficen and the statewide Chancellor. The 
statewide Board of Governors operates more as a coordinating ^ency than as a governing 
body for the colleges. 

Method 

The initial planning for the accountability program called for centralized colleaion and 
reporting of information in five areas: Student access, student success, student saiisfaaion, 
staff composition, and fiscal condition. The purposes of a state level report include 
systemwide planning, policy, and budget formulation. Local colleges would submit the 
required information to the state Boaid, which would then prepare and issue a report. 

The vision of centralized data colleaion and reporting evolved. An eighteen month pilot 
program began in January, 1991 with $375.0(K) in grants awarded to four community 
college distrias in order to produce prototype accountability reports and strategies for 
improving the quality of student outcomes. A task force provid«i consultation to the 
colleges and to -he state Chancellor's Office for the development of a guide for lo<al use 
and a systemwide accountability program. The colleges that were selcaed for grants may 
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not be truly representative of California's 107 local colleges or 71 districts, since they 
possessed exceptional data processing and research capabilities. 

W^ho are the colleges? 

Mt. San Antonio College in Los Angeles County has a single campus serving about 32,000 
students (46% White, 28% Hispanic. 1 1% Asian/PaciBc Islander, and 8% Black). It has 
an Office of Institutional Research that works in cooperation with an Information Services 
Office. Highlights of the Mt. San Antonio project are the development of satisfaaion 
surveys in 29 different services areas and the analysis of census data tapes. 

San Joaquin Delta College has a single campus of about 19,000 students (60% White, 19% 
Asian/Pacific Islander, 15% Hispanic, and 6% Black) in the city of Stockton. An 
Institutional Research and Planning Office conducts research. Delta college has one of the 
most sophisticated tampus data processing systems in California that provides easy access 
and user friendly information services to administrators, faculty, and students. A 
significant feature of Deltas program is the comparison of its performance with that of 16 
other colleges and the state averages on a variety of indicators. 

San Jose-Evergreen is a multi-campus distria with two campuses and a central office. The 
combined enrollment is about 20,000 with a minority of white students and with Latinos 
and Asians making up about 40 percent of the student body. The I>ean of Academic 
Standards and Matriculation is responsible for research. This project embraces the 
development of a detailed demographic data base and the production of a variety of maps 
and reports portraying trends of importance for district planning. 

Santa Barbara City College has one campus virith about 29,000 students (68% White, 21% 
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Hispanic, 8% Asian/Pacific Islander, and 2% Black). A district research committee with 
broad representation throughout the college coordinates resrardi. A major feitutt U the use 
of faculty ndease time to produce reports useful for faculty p'anning and development 
efforts. The projea coordinates with a variety of ingoing projeas, including accreditation 
self-study, development of the statement of institutional direction, ongoing planning 
processes, program review, matriculation, and staff development. 

The coil^e point of view 

The experienas of the colleges were distilled with the help of the task force to produce a 
guide for the initial design of local accountability and institutional effectiveness programs 
(Mills, Amsler, and Kirsch, 1991). A particular goal was to clarify the local perspective 
on accountability. The guide defmes accountability as a way for colleges to prove to 
themselves and others what they are accomplishing and to design ways to monitor and 
ensure continued success. A basic insist was recognizing that the system should serve the 
needs of both the state and the coll^. The state needs information to ass^s achievement 
of the systemwidc mission. The institution needs information to determine where and how 
to improve teaching, learning, and student suppon. The following implementation issues, 
cited from the guide, illustrate the vision of useful local accountability programs — which 
g(M» far beyond the initial idea of centralized data colleaion and reporting towards the 
idea of institutional effeaiveness. 

"Leadership. A strung accountability system requires strong leadership. College 
l^ers must be ».ommitced to its value and importance. Leadenhip is key in 
developing an appri^priate vision of the accounability proems. Leaders must pose 
essential questions >Xlio is responsible for what? What is to be accomplished? What 
nec^aiy resources will be made available? How will the institution know if goals 
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have been met?" 

"Access to information. Access to information is vital to the design of an cflfettivc 
accountability process. In an open sysiem, important data would be shared with all 
sectors of the campus because information is viewed as important to all the 
members of the collie community. ... groups not included in the information- 
access loop cannot be held accountable for decisions that are mad \" 

"Stakeholders. Who should be included in the accountability loop? ... A narrow 
definition of stakeholders can limit the eflfectivcncss of the accountability process. 
Most administrators would agree that buy-in and participation of campus 
stakeholders is desirable. Colleges will differ, however, about who to include and 
to what d^ree information should be shared." 

"Compliance vs. Internsi iJ»e. Reporting must be seen as a tool clearly linked to 
internal decision making. If viewed solely as complying with the demands of 
external agencies, such as the legislature or the Chancellor's Office, then there will 
be no impetus to make links to such vital institutional aaivitics as planning and 
resource allocation. To be motivated to invest enough time to develop an effective 
accountability system, institutional stakeholders must perceive such a system as 
central to the health of their college. They must be convinced that the system will 
not be burdensome to operate and that it will help them fonaion bener." 

Barriers to performance 

The writing of the guide helped to produce a consensus that a Icrally oriented program of 
institutional effectiveness not only satisfies college interests but could also improve the 
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quality of information submitted to the Chancellor's Office. If this is tme. what are the 
main issues in designing an accounubility program for the state? What should the elements 
of a statewide program be? Hayward. Shimasaki, Guthrie, Koppich and Plecki (1991) 
conduaed a study in order to answer these questions. To obtain broad stakeholder 
participation in the survey Hayward et al. interviewed ofBciais of the Chancellor's OflTia, 
the office of the L^islative Analyst, the Department of Finance, the California 
Postsecondary Education Commission, the pilot pn^ram task force and distrios. and 
staff from ten other community college distrias not directly involved in the 
accountability projea — in all rcprcsCTiting 30.4% of the state's community college 
enrollment. 
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Although the study set out to make recommendations regarding the implementation of a 
statewide accountability program, it ended up focusing on a more critical issue - the 
system's readiness for implementation. Although some colleges, with appropriate funding, 
could meaningfully participate in an accountability or institutional effectiveness program, 
many others are not yet ready to do so. Four areas of college need emerged: (1) utilization 
of Management Information Systems (MIS); (2) inarch capability; (3) the patchwork of 
existing accountability requirements; and (4) local perceptions of state requirements. 

The great diversity among colleges in MIS capability has its roots in the highly 
decentralized governance of the California Community College system. Various 
mainframes used by colleges include Hewlen-Packard, UNISYS, Burroughs, and IBM, 
among other less popular brands. The type of software used by colleges depends, of course, 
on the available hardware, and types of application, whether administrative or research, and 
the type of access {dcceniralited "hands-on," or centralized "ai a distance") granted to users. 
Some colleges contract out for data processing services, and others develop in-house 
resources. A number of the visited colleges were not able to supply information required 
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by the state and without additional funding were not prepared to develop their systems. 

Institutional research fiinaicns appeared even more disparate across colleges than 
Management Information Systems. At some colleges research consisted of little moie than 
arraying data for reports, with little emphasis on assessment, analyses or evaluation. In a 
few colleges institutional research was a part of planning and other decision making 
aaivities. Colleges cited state and federally required reports with minimal local utility 
as one reason for lack of sophistication of institutional research offices. The burden of 
meeting these external requirements detracted from investment in local research capacity. 
Ironically, colleges commented that external reporting tended to reduce their access to 
data. State and federal agencies appear unable to provide easy access to data submitted by 
colleges in formats that are locally usefol. and many colleges do not have the resources to 
do it on their own. 

Colleges mentioned the confusion and multiplicity of already existing "accountability " 
requirements. Categorical programs for matriculation assessment, disabled students, the 
economically disadvantaged, vocational education, and for student aid already require 
compliance reponing. These uncoordinated state and federal requirements hamper many 
colleges from pursuing a more comprehensive accountability strategy. 

College staff described various deeply rooted perceptions that mitigated against the 
implementation of a new accountability program. While the study did not examine the 
foundations of these perceptions, Hayward et al. note that people' feelings can influence 
their behavior. 

a) Past promises of access lo submitted data have not l«en kept. 
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b) Colleges are already overburdened with information requests that have no discernible 
useiiil purpose, resulting in a lack of commitment to data quality. 

c) There arc too many uncoordinated duplicate requests for information from various units 
within the Chancellor's Office 

d) There are fears of divisive, inappropriate comparisons between colleges — needlessly 
exacerbating the already difHcult task of managing institutions. 

The state's role 

Despite this dolefiil litany of opposition, Hayward et al. found that colleges tended to see 
the importance of efforts directed at the impnovcmcnt of local programs. Viable 
institutional effectiveness programs are seen as locally useful. Colleges grudgingly conccdv 
that state level accountability is a political necessity, although they would like clearer 
guidance and leadcnhip. The study recommended a program to overcome problems and 
build state and local capacity for accountability. 

1 . Basic data collection and reporting. The Chancellor's Office would annual'y compile 
routinely colleaed information from the statewide Managemrnt Information System and 
other programs. New data colleaion burdens would be minimized. The Chancellor 
would determine the data elements and report formats after consultation with the field. 

2, In-depth accountability studies. Each year the Chancellor's Office would prepare an in- 
dcpih study of one of five areas: student access, student success, stiident satisfaction, faculty 
diversity, and fiscal condition. Colleges and interested state agencies would consult in the 
design of the studies that would produce information for program improvement. For 

o 12 >i 
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example, these studies could serve to identify and help disseminate cxemplaiy practices of 
colleges that operate panicularly successful programs. 

3. Statewide surveys. The Chancellor's Office would periodically conduct sample surveys 
of long and short term student satisfaaion, transfer to senior institutions, and job 
placement. Both state and local officiak have expressed interest in such information, which 
can be very burdensome for colleges to collea on their own. Once developed, the state 
could share these surveys with colleges diat wish to collect more detailed information 
about local conditions. 

4. More effective data colleaion and dissemination. The Chancellor's Office would 
systematically eliminate all duplicative and redundant information requests. The basic 
idea is that colleges should report data one time only and that the Chancellor's OfBce 
should coordinate the use of that information. Further, data transmission should shift from 
the cumbersome circulation of magnetic tapes to a statewide elearonic network. The 
network should facilitate local access to submined information. 

5. Depository/clearinghouse. The Chancellor's Office would serve as a depository for 
exemplary accountability models and provide technical assistance to colleges as they 
establish their own programs. In addition to providing technical assistance documents and 
staff development services, this office would help to link up colleges who need assistance 
with other colleges that can share their expertise. 

A statewide indicators report 

Table I displays the oudine of a statewide indicators report thai gradually emerged 
through dialogue with die colleges, die task force, representatives of interested state 
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Table 1: Indicatois for Statewide Community GiUege Accoimtability Reponing 





Context 


INPUT 


PROCESS 


OUTCOME 


STUDENT Access 


General enrollment 

Transition from High 
School 


Financial aid recipients 


Categorical programs 

Basic skills and ESL 
eniotlment 




Student Success 








Persistence 

Completion 

Transfer to four yrar 
institutions 

Job placement 


Student 
Satisfaction 








Satisiaaion with: 

• access 

• instruaion 

• services 

• facilities 


Staff Composition 






Faculty demographics 

Full- or part-time 
(kculty assignment 




FISCAL Condition 


Federal, state and local 
frinding 

FisoJ stability 
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agencies, and community college researchers. The sclcaion of performance indicators was 
an essential part of the accountability program. Work done by Oakes (1986), Mumane 
(1987), and National Center for Education Statistics (1988, 1991b) helped to fomish 
starting points. The major principle underlying the design of the report follow. 

a) An indicator is a meas'irable characteristic of the condition of the community colleges 

b) Restrictions on the number of indicators are suggested by the burden associated with the 
processing and use of information 

c) Indicators should represent the charaaeristics of colleges that are related to desired 
student outcomes 

d) The indicators are primarily of statewide interest. Colleges at their discreuon may 
augment the list to provide more usefol descriptions of local conditions 

c) Appropriate background information helps to interpret outcomes, i.e., context (the 
environment in which colleges operate); input (fiscal resources and policies); and processes 
(plans and aaivities to meet student needs). 

VC^hai does it mean? 

This study b^ by asking what "accountability" means. The main finding of this study is 
that accountability in this context has two meanings. At the state level it ties to 
educational finance, competition for funds, and the desire to make policy based on 
objective, defensible information. At the local, college Icvc? it refers to institutional 
effectiveness. 

IK 
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The concext for defining ihc ni(»niiig of accountability in California is one of shrinking 
and more centralized funding. A 1987 study by Policy Analysis for California Education 
(PACE) described California's school finance in die early 1980s was "unstable and 
uncertain." The tax reform measures in the 1970s cut property taxes in half and severely 
limited the ability of sute and local gpvemments to expend revenues. These fiscal 
measures tended to centralize educational fmance at the state level. Funding in constant 
dollars declined and student enrollments incr^sed. Ori>^n by entidements and polidc^ 
necessity ether government programs, e.g., prisons, social, and medical service, absorbed 
ever larger percentages of available funds. 

Educational accountability could be used to support for increased funding for sdiools and 
colleges and make decisions. (Hayward, 1991} 

"State level interviewees tended to view a useful accountability system as one which 
would assist them in arriving at three types of allocation recommendations 
regarding: 1) the distribution of increasingly scarce state resources between 
competing segments, 2) taigering funds on programs judged to be relatively more 
efifecnve than others, and 3) providing a rationale for seeking additional funds. 



One set of questions faced by state control agencies revolves around the 
determination of the optimal mix of state funds among the various providers of 
education. Should the state pay for remedial cou»es at the University of California 
or only fund those programs at the community college and high school adult level? 
Can adult education courses be more ci»t-effectivcly offered by high school adult 
education programs or community college programs? Is the current allocation of 
vocational education funds optimal or should a higher percentage of the funds flow 
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to the institutions which can most cost-cffcctivcIy offer vocational programs? 

Another set of questions revolves around decisions about program offerings within 
segments. Would the state be better served by allocating more dollars for 
matriculation or for EOPS? Should there be additional state incentives provided 
for establishing more effective transfer programs or for investing in "tech-prep" 
vocational programs? These arc all important questions which have frustrated 
budget analysts for years and cause them to seek better information to inform these 
critical decisions. 

A third set of questions centers on the frustration esqjrKsed by the Chancellor and 
his stafifwho argue that an cffoaive statewide accountability system would enable 
his office to provide strong rationale for the budget requests to die Governor and 
the legislature for additional funds." 

The education finance issues demonstrate an input oriented rationale for accountability. By 
contrast, a resulted oriented stimulus for institutional effeaiveness is found in public 
concern over student outcome. California, along with many other sutes and the federal 
government, has harbored die perception of a troubled education system. (Feder, 1986; 
Office of Educational Research and Improvement, 1988; Mirga, 1989; Kaagan and Colcy, 
1989). Although this mentality has traditionally focused on supposed deficits in K-12 
education, dicrc is a growing consensus that postsecondary institutions also need attention. 
(National Center for Education Statistics, 1991; Albert, 1991; Ewcll. 1991) For 
example, goal five of the Bush administration's America 2000 Program penains dirccdy 
to higher education: "Every adult American will be literate and will possess die knowledge 
and skills necessary to compete in a global economy and exercise the rights and 
responsibilities of leadership." This goal is attainable in part through "better and more 
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accountability. ** Higher education is pan of a larger system whose needs require more than 
aaion in only the K- 12 segment. The resuhs of the pilot program suggest that the way to 
improve student outcomes is locally run institutional ef&aiveness programs. 



How might institutional effeaivcness programs work to improve student outcomes? A 
systemic approach (c.g, Kaagan and Coley, 1989) would specify a model with major 
goals, pr<^rams, instructional methods, and resources. Use the model to identify 
imponant obstacles to success, leverage points for change^ and to devi^ indicators that 
describe the limaioning of the model. Try out difierent strategics and monitor the 
indicators to gauge effectiveness. Use the feedback to refine the model and strategics for 
improvement. 

Although this approach has intuitive appeal, the results of the pilot program suggest that it 
leaves out the fector of stakeholder panicipation. The traditional control of curriculum 
and instruction by faculty indicates that faculty groups should have a voi«» in developing 
institutional effectiveness programs. Students, and trustees as well as administraton need 
to be involved to the extent that the success of the institutional effectiveness program 
depends on their coof^ration, commitment, or acceptance. These findings appear 
consistent with Bickman and Peterson's (1990) observation that different stakeholdere - 
including accrediting agency representatives, trustee, administrators, faculty, students, 
business or community leaders have differing ideas about which outcomes are most 
desirable and how they should best be pursued. MufFo's (1990) national survey of stams of 
student outcomes assessnieru at colleges and universities spears to confirm this finding. 
"Involving the feculty in ihe process is seen as critiad to overcoming initial skepticism, 
resistance, and fear." 



A number of features of ihe stare accountability program are similar to recommendations 
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sometimes made in Total Quality Management (TQM) programs: the involvement of 
stakeholders; strong leadership for improvement of teaching and learning; developing 
higher quality and more uscfiii data systems; making decisions based on objcaive 
information; and interest in customer (e.g., student, faculty) satisfeaion. (Marchcsc, 1991) 
Despite die interest in these features, colleges did not embrace the idea of TQM as an 
institutional effcaivencss strategy. Informal discussions unearthed specific reservations: the 
busines jargon sounded alien; TQM might just be another "here today, gpne tomorrow" 
fed; and doubt that TQM was consistent with traditions of academic freedom for faculty. 
While some permutation of the TQM philosophy may become popular in academic 
settings, this study suggests that some accommodation from business environments to 
academic settings will facilitate its acceptance by educators. 

Summing up 

If concern about the quality of student outcomes abides, and the competition for public 
funds remains unabated, there will likely be enduring legislative interest in accountability 
and institutional eflfcaivencss programs. It is difficult to fault desires to make better 
public policy decisions using appropriate information, or to continuously improve the 
quality of teaching and l«uning. A risk is that reporting of a few numben or test scores 
will foster a one-dimensional, unrealistic view of education. Perhaps the thoughts about 
physics of Niels Bohr, the Nobel prize winning scientist can provide some guidance. Bohr 
remarked that die task of physics is not to find out how nature is. Physics concerns what wc 
can say about nature. And the best we can say is always partial and incomplete only by 
entertaining multiple and mutually limiting points of view, building up a composite 
piauie, can wc ^roach the real richness of the world. 
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